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INTRODUCTION 

Graduate students training not only needs to cultivate 
their theoretical literacy, but also improves their ability to 
complete scientific research tasks. The teaching purpose of 
methodology course is to help graduate students learn and 
understand standardized research methods. In recent 
years, in order to improve the research ability of 
postgraduates, many universities such as Tsinghua 
University and Nanchang University have _ offered 
methodology courses in the postgraduate training 
programs. Evaluating the classroom teaching quality of 
methodology courses is helpful to improve the classroom 
teaching and improve the teaching quality. 


1. Theresearch status 

In the research of existing methodology courses, most 
scholars focus on the importance and significance of the 
existence of methodology courses. Such as Zhao 
Yuxin(2015) It is believed that the talents trained by 
higher education should be talents with both professional 
knowledge and creative thinking. Introducing research 
methodology teaching in higher education is conducive to 
helping college students build a scientific research ability 
system, but there is still a big lack in the training of 
research methodology in China, which can be seen from 
students’ research topics, literature research and 
dependence on tutors; Yudengk(2016) It is believed that 
setting up methodology courses can help students master 
research methods scientifically and systematically; 
Sunshibing(2014) and other scholars believe that entering 
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the methodology in the postgraduate education stage will 
help cultivate their ability to handle events correctly and 
effectively; Wujiannan(2014) and other scholars found 
that Britain and the United States attach importance to the 
cultivation of practical ability of postgraduates, and both 
set methodology courses as compulsory subjects for 
postgraduates, and attach importance to the training of 
students’ research methods to improve the quality of 
students’ cultivation. Foreign universities attach 
importance to such courses, which is worth learning. 


As can be seen from Table 2 and Figure 1, the key research 
direction of classroom teaching quality evaluation in China 
at present is mainly to construct the quality evaluation 
index system of classroom teaching buildings for college 
courses. When constructing the evaluation index system, 
most scholars will choose students as the main body for 
evaluation from the perspective of students. 


Different scholars have different evaluation index systems 
according to different curriculum characteristics. Such as 
Wang Junming(2020) and other scholars have finally built 
a Classroom teaching quality evaluation index system with 
teaching scheme, strategy and effect as the first-class 
indicators through Delphi method; Qiuwenjiao(2016) and 
other scholars have made many revisions through multi- 
party consultation, and constructed inquiry-based 
classroom teaching evaluation indicators from three 
aspects: teachers, students and inquiry effects; Wu 
Guoyu(2015) and other scholars revised the _ initially 
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constructed indicators by issuing questionnaires, and teaching evaluation and feedback based on the basic 
finally built an index system including seven first-level requirements of the curriculum for talent training and 
indicators: teaching enthusiasm, teaching content, teaching process; Yue Qi and Wen Xin(2018) An index 
learning value, and homework assessment. system including teachers’ quality, teaching attitude, 
Zhaoxinrui(2019) and other scholars use the analytic teaching content, teaching method and teaching effect was 
hierarchy process to build a six-dimensional evaluation constructed. 


index system including teaching team, teaching activities, 


2. The preliminary construction of teaching quality evaluation indicators 

The purpose of teaching quality evaluation is to find out the shortcomings in teaching and further improve it, so that the 
teaching quality can be improved and the task of student training can be completed better. According to the customer 
perceived quality model, in order to comply with the customer-centered concept, manufacturers should evaluate the 
quality of goods from the customer's point of view. This model is also applicable to the evaluation of teaching quality. The 
Starting point of curriculum teaching is to serve students and take students as the center. Therefore, when evaluating 
classroom teaching quality, students should be asked to evaluate it. Combined with previous studies, this paper will draw 
up the evaluation index of classroom teaching quality of methodology courses from three dimensions: teaching process, 
teaching method and teaching effect. 


Table 1 Preliminary evaluation index table of classroom quality of methodology courses 
uta Secondary 
index index 





Three-level index 


C1 has a high degree of interaction between teachers and students, and students can 
Bi classroom | havea sense of participation in the classroom 
atmosphere C2 classroom learning atmosphere is strong, and teaching auxiliary facilities and 


tools are complete 


C3 teaching content is properly connected with undergraduate course without 
obvious fault 


C4 teaching content is focused, hierarchical and easy for students to master 


process ante systematic thinking system 
C6 combines theoretical knowledge to discuss the latest research results and 


Guidance C8 guides students to train related research methods after class 


C7 discusses the teaching contents of methodology courses according to topics or 
shows cases 

under lesson , ; a 

B3 C9 guides students’ problems in practical application after class 


BA divareit C10 A variety of teaching methods are combined, and the appropriate method is 
y selected according to the needs of the teaching content 


C11 adopts the enlightening method, which allows students to explore and discover 
Be suidance themselves and improve their creativity 
A2 teaching 6 C12 adopts the method of discussion, which enables students to inspire each other, 


method think independently and get rid of dependence 


C13 adopts example teaching to make students understand how to apply research 
B6 practicality | methods in practical cases 


C14 adopts practice teaching and carries out relevant exercises according to the 
content of methodology course 
C15 can well absorb and master the teaching content of this course 
harvest C16 is able to apply the theoretical knowledge to practical problems 
A3 teaching C17 can master the standard research methods and complete scientific research 
effect B8 capacity tasks independently 


improvement | C18can forma scientific thinking system and form its own thinking on relevant 
issues 


3. Research methods 

3.1. Delphi method 

Delphi method is a non-interference way of discussion. The experts concerned are organized into an expert group and 
members of the group are guaranteed to have "back-to-back" discussions on the same issue.The steps of Delphi method 
are as follows: the research group formulates the questionnaire according to the research content and sends it to the 
expert group; the expert group discusses, and the research group summarizes and analyzes the discussion results, and 
then sends it to the expert group. Repeat the above steps until the coordination of experts, that is, the consensus reaches a 
certain standard, stops, and takes the final discussion result as the prediction result. 


3.2. Entropy weight method 
The method of using entropy weight method to determine the index weight has been widely used in the research of index 
system construction. 
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First, standardize the values of the data. Assuming that K indicators are given, the standardized value of each index data is 
_ Xjj—min(X;) 
ij max(X;)—min(X;) 


Secondly, according to the standardization results obtained in the first step, the information entropy is calculated. 
According to the definition of information entropy in information theory, the information entropy of a group of data is 


E; = —In(n)™ ea pij Inpi;. 


amongp;; = SRT ifp;; = 0, then definelimy, =o piyinpi; = 0. 


n 
Disa 


Third, determine the weight of each index.According to the formula: 


w, = 27" GG =12.....k 
i~yo—ye t= jpemenle) 


Calculate the weight of each index, whereE,Is information entropy. 


4. Research design 

4.1. Determined by experts 

The experts selected in this study are teachers of methodology courses in Colleges and universities, with rich teaching 
experience and knowledge reserve. When Delphi method is used for expert consultation, the determination of the expert 
group is best controlled within 20 people. Considering the factors such as the research topic in this paper, 15 people are 
finally selected to form the expert group. 


4.2. Expert consultation 

4.2.1. Basic information of experts 

The number of experts in this study was determined to be 16.[n terms of gender distribution, there are 6 men and 10 
women; in terms of professional titles, there are 5 professors, 3 associate professors and 8 lecturers. 


4.2.2. Enthusiasm of experts 

Referring to the previous literature, this study will use the questionnaire recovery rate as an indicator to reflect the 
enthusiasm of experts. The higher the questionnaire recovery rate is, the more active the experts are.In the first round of 
consultation, questionnaires were distributed to 16 members of the group, and one person did not return the 
questionnaire. A total of 15 valid questionnaires were collected. The recovery rate of the first round of questionnaire was 
93.75%; in the second round of consultation, all the questionnaires were collected and the recovery rate reached 100%.It 
can be seen from the above that the recovery rate of the two rounds of questionnaires is at a high level, which indicates 
that the members of the expert group actively cooperate with the consultation and have a high degree of enthusiasm. 


4.2.3. Expert authority 

In this study, the authority coefficient (CR) is used to measure whether the members of the expert group have certain 
authority to ensure the credibility of the consultation results.The calculation formula of the authority coefficient (CR) is Cr 
= (Ca + CS) / 2, in which ca is the basis for experts to score each index in the index system, and CS is the familiarity of 
experts with the research problem.Ca and CS are assigned on the following basis: 


Table 2 evaluation basis of experts for this research index 
Degree of evidence 
large in Small 


Judgment basis 


Table 3 assignment table of experts' familiarity with this research index 
Familiarity Veryunfamiliar beunfamiliar with common _ be familiar with Very familiar 








| evaluation | 0] | SB 


According to the assignment basis, the calculation result of Cr is 0.89, which indicates that the experts selected in this 
study have a high degree of authority and the consultation results have credibility. 


Table 4 Authority degree of experts 
Judgment basis Ca_ Familiarity Cs Authority degree Cr 


0.85 0.89 





4.2.4. Degree of expert coordination 

In this study, the coefficient of variation (CV) and Kendall coefficient (W) are used to measure the coordination degree of 
experts. Coefficient of variation is the ratio between the average difference and the average value of each index, which is 
used to measure the coordination degree of experts to each index. The smaller the value of CV, the better the coordination 
of experts, and the general coefficient of variation (CV)<0.25, the better the coordination degree of this index, that is, the 
consensus among the members of the expert group is higher. 
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The calculation formula of Kendall coefficient (W) is: W = Te ‘= d;, where m is the consulting expert n is the 
number of evaluation objects, iii, d? = ae Cy, — M,;)*o sj = pes Rij, R,jScore the j-th index for the i-th expert, S; 


i 


is the sum of the scores of the j-th index;M,; = = es S; arepresents the arithmetic mean of the sum of the j-th index 


scores. 


When different experts have the same score for a certain index, W needs to be corrected. The correction formula 


Sohal 12 = 
isW = 74 Dez 
m?2(n3—n)—m Dit, Ti dj=1 


given by the i-th expert, and t; represents the number of the same level of the i-th expert in the L group. 


L 
d; where T; = Dee —t;) ,L represents the number of groups with the same score 


By calculating the coefficient of variation of each index, it can be seen that the CV value of each index finally determined in 
this study is less than 0.25. At the same time, the Kendall coefficient of each index is calculated by SPSS 25.0. The results of 
the first round of expert consultation are: W=0.214, and P<0.001, which is obviously significant; The result of the second 
round of expert consultation is: W=0.256, and P<0.001, which is obviously significant. From the above results, we can see 
that the Kendall coefficient of the two rounds of expert consultation is at a higher level, and the Kendall coefficient of the 
second round of expert consultation is higher. 


According to the above calculation results of CV and W, it can be seen that the consensus of experts is high, so the 
discussion will be stopped and the third round will not be conducted. 


Table 5 Expert coordination coefficient table 
Kendall W_ chi-square P value 


0.214 54.523 <0.001 





0.256 61.374 | <0.001 


5. the construction of index system 
Drawing lessons from the previous practice, delete the indicators in the first round of consultation with the index score =4 
points and the number of people less than 80%, after finishing, the indexes C2, C3 and C10 are deleted. 


After deleting the unsatisfactory indicators, based on the revised opinions of experts sorted out in the two rounds of 
consultation, this study revised the initial indicators as follows: First, some experts believe that methodology courses pay 
attention to cultivating students’ thinking, so the classroom atmosphere in which students find and ask questions is an 
important feature of methodology courses. This study recognizes this view and adds it to the secondary indicator 
“classroom atmosphere’; Secondly, some experts believe that it should be in line with the professional direction. Since the 
main goal of graduate methodology courses is to cultivate the research ability of graduate students, attention should be 
paid to the integration of courses and majors. This study adopts this suggestion and adds it to the secondary index 
"teaching content"; Third, some experts pointed out that students should be instructed to conduct in-depth research on 
teaching content. This study adopts this suggestion and adds this index to the secondary index "practicality"; Fourthly, 
some scholars think that index C13 is duplicated with index C7, and the content of index C13 does not conform to the 
dimension of "teaching method", so index C13 should be deleted, and this opinion is adopted in this study. Fifthly, some 
scholars pointed out that some indicators in this study were unclear, so this study modified some indicators in terms of 
language expression according to experts’ suggestions. 


According to the score value of each index, the entropy weight method is used to calculate the weight of each index, and 
finally the index system is determined as shown in the following table: 


Table 6 Evaluation Index System of Classroom Teaching Quality of Methodology Courses 
Secondary 
index 


C1 has a high degree of interaction between teachers and 
B1 0.144 students, and students can have a sense of participation in the 
classroom classroom 


Primary 


weight Three-level index 
index 


weight 


atmosphere C2 students actively explore the connection between teaching 


content and their majors 


C3 teaching content is in line with students’ own professional 
direction 


At C4 teaching content is focused, hierarchical and easy for students 
‘achin to master 
6 B2 teaching The content structure of C5 is systematic, which helps students to 
process 0.782 / ie ae 
content establish a systematic thinking system 
C6 discusses the teaching content of methodology course 
according to the topic or the case shows the dynamic discussion 


C7 combines theoretical knowledge, the latest research results 
0.097 
and research of disciplines 
Guidance ors C8 guides students to carry out course-related training after class | 0.34 
0.073 


under C9 gives guidance and feedback to students’ problems and 
einai 0.659 
lesson B3 performance in practical application after class 
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C11 adopts the enlightening method, which allows students to 
explore and discover themselves and improve their creativity 


A2 Be eeiGance: | Vises C12 adopts the method of discussion, which enables students to 0.235 
teaching | 0.278 inspire each other, think independently and get rid of dependence | ~ 


method 


practicality 


effect B7 ability 


improveme 


6. DISCUSSION 

In recent years, the number of graduate students has been 
increasing continuously. In order to ensure the steady 
improvement of their training quality under the general 
environment of graduate enrollment expansion, 
universities attach great importance to the training quality 
of graduate students. Methodology courses can help 
students master scientific and systematic research 
methods, help students better understand relevant 
research and complete scientific research tasks. Therefore, 
improving the classroom quality of methodology courses 
is the direction that universities should strive for at this 
Stage. In this paper, Delphi method is used to discuss the 
classroom quality evaluation indexes of methodology 
courses, and finally an index system including 3 first-class 
indexes and 17 second-class indexes is constructed. 
Methodology courses can also use this index system to 
evaluate the teaching quality, which can further improve 
the index with low scores and contribute to the high- 
quality construction of methodology courses in colleges 
and universities. 
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